Task Learning through Visual Demonstration and Situated Dialogue
نویسندگان
چکیده
To enable effective collaborations between humans and cognitive robots, it is important for robots to continuously acquire task knowledge from human partners. To address this issue, we are currently developing a framework that supports task learning through visual demonstration and natural language dialogue. One core component of this framework is the integration of language and vision that is driven by dialogue for task knowledge learning. This paper describes our on-going effort, particularly, grounded task learning through joint processing of video and dialogue using And-Or-Graphs.
منابع مشابه
The Effects of Task Variation on the Accuracy and Complexity of Iranian EFL Learners’ Oral Performance
Task variation is an integrative method aiming at the importance of learner-to-learner interactions in a wide range of learning contexts and fostering authentic use of language and meaningful communication. This study investigated the impact of task variation on the accuracy and complexity of Iranian EFL learners’ oral speech. In so doing, 80 intermediate EFL learners, majoring English at the I...
متن کاملA Wizard-of-Oz Environment to Study Referring Expression Generation in a Situated Spoken Dialogue Task
We present a Wizard-of-Oz environment for data collection on Referring Expression Generation (REG) in a real situated spoken dialogue task. The collected data will be used to build user simulation models for reinforcement learning of referring expression generation strategies.
متن کاملThe Relationship between Willingness to Communicate and Two Types of Speaking Task: Narrative and Dialogue
This study attempted to examine the relationship between WTC and two types of speaking tasks:Dialogue and narrative/monologue. The significance of this study lies in its theoreticalcontributions to the WTC construct and the pedagogical implications in second languageteaching and learning. Sixty one intermediate female EFL learners were selected through anOxford Placement Test (OPT) from Tabares...
متن کاملOptimizing Situated Dialogue Management in Unknown Environments
We present a conversational learning agent that helps users navigate through complex and challenging spatial environments. The agent exhibits adaptive behaviour by learning spatiallyaware dialogue actions while the user carries out the navigation task. To this end, we use Hierarchical Reinforcement Learning with relational representations to efficiently optimize dialogue actions tightly-coupled...
متن کاملIncremental Generation of Visually Grounded Language in Situated Dialogue (demonstration system)
We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor (Yu et al., ). The system integrates an incremental, semantic, and bidirectional grammar framework – Dynamic Syntax and Type Theory with Records (DS-TTR1, (Eshghi et al., 2012; Kempson et al., 2001)) – with a set of visual classifiers that are learned throughout the intera...
متن کامل